This paper develops methods for proving Lyapunov stability of dynamical systems subject to disturbances with an unknown distribution. We assume only a finite set of disturbance samples is available and that the true online disturbance realization may be drawn from a different distribution than the given samples. We formulate an optimization problem to search for a sum-of-squares (SOS) Lyapunov function and introduce a distributionally robust version of the Lyapunov function derivative constraint. We show that this constraint may be reformulated as several SOS constraints, ensuring that the search for a Lyapunov function remains in the class of SOS polynomial optimization problems. For general systems, we provide a distributionally robust chance-constrained formulation for neural network Lyapunov function search. Simulations demonstrate the validity and efficiency of either formulation on non-linear uncertain dynamical systems.

The Position Embedding (PE) is critical for Vision Transformers (VTs) due to the permutation-invariance of self-attention operation. By analyzing the input and output of each encoder layer in VTs using reparameterization and visualization, we find that the default PE joining method (simply adding the PE and patch embedding together) operates the same affine transformation to token embedding and PE, which limits the expressiveness of PE and hence constrains the performance of VTs. To overcome this limitation, we propose a simple, effective, and robust method. Specifically, we provide two independent layer normalizations for token embeddings and PE for each layer, and add them together as the input of each layer's Muti-Head Self-Attention module. Since the method allows the model to adaptively adjust the information of PE for different layers, we name it as Layer-adaptive Position Embedding, abbreviated as LaPE. Extensive experiments demonstrate that LaPE can improve various VTs with different types of PE and make VTs robust to PE types. For example, LaPE improves 0.94% accuracy for ViT-Lite on Cifar10, 0.98% for CCT on Cifar100, and 1.72% for DeiT on ImageNet-1K, which is remarkable considering the negligible extra parameters, memory and computational cost brought by LaPE. The code is publicly available at https://github.com/Ingrid725/LaPE.

Weakly supervised semantic segmentation is typically inspired by class activation maps, which serve as pseudo masks with class-discriminative regions highlighted. Although tremendous efforts have been made to recall precise and complete locations for each class, existing methods still commonly suffer from the unsolicited Out-of-Candidate (OC) error predictions that not belongs to the label candidates, which could be avoidable since the contradiction with image-level class tags is easy to be detected. In this paper, we develop a group ranking-based Out-of-Candidate Rectification (OCR) mechanism in a plug-and-play fashion. Firstly, we adaptively split the semantic categories into In-Candidate (IC) and OC groups for each OC pixel according to their prior annotation correlation and posterior prediction correlation. Then, we derive a differentiable rectification loss to force OC pixels to shift to the IC group. Incorporating our OCR with seminal baselines (e.g., AffinityNet, SEAM, MCTformer), we can achieve remarkable performance gains on both Pascal VOC (+3.2%, +3.3%, +0.8% mIoU) and MS COCO (+1.0%, +1.3%, +0.5% mIoU) datasets with negligible extra training overhead, which justifies the effectiveness and generality of our OCR.

在本文中，我们重新审视了从单线图中自动重建3D对象的长期问题。以前的基于优化的方法可以生成紧凑而准确的3D模型，但是它们的成功率在很大程度上取决于（i）确定一组真正的真正几何约束的能力，以及（ii）为数值优化选择一个良好的初始值。鉴于这些挑战，我们建议训练深层神经网络，以检测3D对象中几何实体（即边缘）之间的成对关系，并预测顶点的初始深度值。我们在大型CAD模型数据集上进行的实验表明，通过利用几何约束解决管道中的深度学习，基于优化的3D重建的成功率可以显着提高。

虽然视觉变压器（VT）体系结构在计算机视觉中越来越流行，但纯VT模型在微小的数据集上的性能较差。为了解决这个问题，本文提出了改善小型数据集VT性能的地方指南。我们首先分析，由于VTS中自我注意的机制的高灵活性和内在的全球性，因此很难用有限的数据来学习局部信息，这对于理解图像非常重要。为了促进本地信息，我们通过模仿已经训练有素的卷积神经网络（CNN）的特征来实现VT的当地指南，灵感来自CNN的内置本地到全球层次结构。在我们的双任务学习范式下，由低分辨率图像训练的轻型CNN提供的局部指导足以加速收敛并在很大程度上提高VT的性能。因此，我们的本地指导方法非常简单有效，可以作为小型数据集中VT的基本性能增强方法。广泛的实验表明，我们的方法在小型数据集中从头开始训练时可以显着改善VT，并且与不同种类的VT和数据集兼容。例如，我们提出的方法可以将各种VT在微型数据集上的性能提高（例如，DEIT 13.07％，T2T为8.98％，PVT为7.85％），并使更强大的基线PVTV2提高了1.86％至79.30％，显示出来小型数据集上的VT潜力。该代码可从https://github.com/lkhl/tiny-transformers获得。

在本文中，我们表明样品的欧几里得规范的差异可以在空间翻译和划分归一化之后对语义差异甚至混乱做出贡献。为了解决这个问题，我们提出了一种直观但有效的方法，以均衡样品向量的欧几里得规范。具体来说，我们$ l_2 $ - 在批准之前将每个样品向量归一化，因此样品向量的幅度相同。由于所提出的方法结合了$ L_2 $归一化和批量归一化，因此我们将我们的方法称为$ L_2 $ bn。 $ l_2 $ bn可以增强阶层内特征的紧凑性，并扩大阶层间特征的差异。此外，它可以帮助梯度收敛到稳定的量表。 $ l_2 $ bn易于实现，并且可以在没有任何其他参数和超参数的情况下发挥其效果。因此，它可以用作神经网络的基本归一化方法。我们通过对图像分类和声学场景分类任务进行各种模型的广泛实验来评估$ L_2 $亿美元的有效性。实验结果表明，$ L_2 $ bn能够提高各种神经网络模型的概括能力，并取得了可观的性能改进。

随着社交媒体平台越来越多地采用了简短的视频，通过视频帖子减少错误信息的传播已成为社交媒体提供商的关键挑战。在本文中，我们开发了在社交媒体帖子中检测错误信息的方法，从而利用了视频和文本等方式。由于缺乏在多模式数据集中检测错误信息检测的大规模公共数据，因此我们从Twitter收集160,000个视频帖子，并利用自学学习的学习来学习联合视觉和文本数据的表达性表示。在这项工作中，我们提出了两种新方法，用于基于对比度学习和掩盖语言建模的短形式社交媒体视频帖子中的语义不一致。我们证明，我们的新方法在通过随机交汇正面样本和在野外的新手动标记测试集中，在野外生成的人工数据上的最新方法都超过了当前的最新方法，以进行语义错误信息。

在现实生活中，每个人都在一定程度上表现出来，期望人们在互联网上表现自己更加困难，因为仍有很少的检查或后果，用于向他人张贴有毒的东西。然而，对于另一方的人来说，有毒文本往往导致严重的心理后果。检测这些有毒文本是挑战性的。在本文中，我们试图使用CNN，Naive Bayes Model以及LSTM等机器学习方法构建毒性探测器。虽然他人占据了许多基础工作，但我们的目标是建立提供比前辈更高的准确性的模型。我们使用LSTM和CNN制作了非常高的精度模型，并将其与语言处理中的去解决方案进行了比较，朴素的贝叶斯模型。嵌入方法也适用于赋予我们模型的准确性。

Process monitoring and control are essential in modern industries for ensuring high quality standards and optimizing production performance. These technologies have a long history of application in production and have had numerous positive impacts, but also hold great potential when integrated with Industry 4.0 and advanced machine learning, particularly deep learning, solutions. However, in order to implement these solutions in production and enable widespread adoption, the scalability and transferability of deep learning methods have become a focus of research. While transfer learning has proven successful in many cases, particularly with computer vision and homogenous data inputs, it can be challenging to apply to heterogeneous data. Motivated by the need to transfer and standardize established processes to different, non-identical environments and by the challenge of adapting to heterogeneous data representations, this work introduces the Domain Adaptation Neural Network with Cyclic Supervision (DBACS) approach. DBACS addresses the issue of model generalization through domain adaptation, specifically for heterogeneous data, and enables the transfer and scalability of deep learning-based statistical control methods in a general manner. Additionally, the cyclic interactions between the different parts of the model enable DBACS to not only adapt to the domains, but also match them. To the best of our knowledge, DBACS is the first deep learning approach to combine adaptation and matching for heterogeneous data settings. For comparison, this work also includes subspace alignment and a multi-view learning that deals with heterogeneous representations by mapping data into correlated latent feature spaces. Finally, DBACS with its ability to adapt and match, is applied to a virtual metrology use case for an etching process run on different machine types in semiconductor manufacturing.

Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time-varying covariates. In mobile health applications, these covariates are typically collected at different frequencies over a long time horizon. In this paper, we propose a deep spectral Q-learning algorithm, which integrates principal component analysis (PCA) with deep Q-learning to handle the mixed frequency data. In theory, we prove that the mean return under the estimated optimal policy converges to that under the optimal one and establish its rate of convergence. The usefulness of our proposal is further illustrated via simulations and an application to a diabetes dataset.